Associations between digital health literacy and health system navigating abilities among Norwegian adolescents: validating the HLS19-DIGI scale using Rasch modeling

Background Despite increasing global attention to health literacy and adolescents’ digital health information seeking, no unidimensional instruments measuring digital health literacy (DHL) in adolescents have reportedly been validated using Rasch modeling. Moreover, the evidence of adolescents’ abilities to navigate the health system (NAV-HL) in light of their DHL proficiency is still scarce. Therefore, our study aims to evaluate the psychometric properties of a DHL instrument (HLS19-DIGI scale) in order to investigate DHL in adolescents and young adults aged 16–25 and associations with abilities to navigate the health system. Methods A population-based cross-sectional survey among 890 Norwegian adolescents was conducted during April–October 2020 using computer-assisted telephone interviewing. Rasch modeling, independent samples t-test, chi-square test, and binary regression models were used to analyze the data. Results The HLS19-DIGI scale was sufficiently unidimensional, whereas no differential item functioning or disordered response categories were observed. However, relatively poor targeting was revealed indicating too many easy items for the target population. Yet, a high proportion (54%) of low DHL proficiency in adolescents was observed, as well as DHL was positively associated with the abilities to navigate the health system. Conclusions The HLS19-DIGI scale is considered a sufficiently unidimensional and valid instrument for measuring DHL in adolescents, which may be a useful tool for health authorities, public health workers, and health service providers. While DHL affects adolescents’ abilities to navigate the health system, future research should measure and examine their ability to utilize digital health services, separately. Supplementary Information The online version contains supplementary material available at 10.1186/s12889-024-19405-w.


Introduction
In several Western countries, adolescents are increasingly expected to take responsibility for their own health [1].They are exposed to a wealth of health information [2], while research has suggested that their general health literacy (GHL) in terms of their abilities to access, understand, critically appraise, and apply such information may be insufficient [3].Correspondingly, a conceptualization of access to healthcare by Levesque et al. [4] refers to five relevant abilities of the population: (1) to perceive, (2) to seek, (3) to reach, (4) to pay, and (5) to engage.These abilities largely reflect the health system navigating skills (item content) in the Navigation Health Literacy (NAV-HL) scale of the WHO Action Network on Measuring Population and Organizational Health Literacy (M-POHL) [5][6][7].Furthermore, ever-increasing digital transformation in healthcare should also warrant policymakers, practitioners, and researchers to acknowledge the importance of understanding people's capabilities in using these resources and technologies for maintaining and/or promoting their health.
In addition to GHL and NAV-HL, digital skills are needed due to continuing digitization of health information and the healthcare services.These skills, called digital health literacy (DHL), are defined by the HLS 19 -Consortium of M-POHL as "the ability to search for, access, understand, appraise, and apply online health information, the ability to formulate and express questions, opinion, thoughts, or feelings when using digital devices" [6, p.278].While knowing that the terms DHL and eHealth literacy might be used interchangeably in research, we prefer the term DHL in accordance with the HLS 19 -Consortium of M-POHL [6, p. 278].Specifically, the term digital health is more inclusive by means of their close relationship to mHealth (mobile) and artificial intelligence, and other emerging areas of innovation and information technology.DHL, including media health literacy and eHealth literacy throughout the lifespan have been explored in a number of studies; in childhood and adolescence [7][8][9] and during adulthood included the elderly [10,11].Moreover, it has also been studied from the aspect of cultural transition [12], health conditions [13], and health behaviors [14].While research suggest that there is a correlation between GHL and DHL [8,9], it is also acknowledged that quality assured digital health information can be used to promote healthy lifestyles and help prevent physical and mental illness over time [10][11][12].Such information services can be part of a broad-based public health program commencing early in the life course and continuing throughout the lifespan.For instance, through school, adolescents are well served by the Internet.However, such information is only useful for those who are capable to seek, understand, appraise, and apply the accessed information effectively [13].
Acknowledging that these abilities among adolescents may be insufficient [3], mainstreaming access to such information, therefore, can enable young users to better understand their own or relatives' health and illnesses, and thereby empower patients in their interactions with health professionals.Thus, people who are capable of using digital health information for making decisions about their own health are expected to be more resourceful from a life course perspective.Accordingly, this is what we refer to as a component of DHL that involves the skills required for using search engines, mastering search strategies and critically appraising sources and identifying relevant and valid digital health information [5].
During adolescence, people are more likely to develop addictive behavior [14] and mental illness [15][16][17] as well as they are less likely to use health promoting and disease preventing services comparing to adults [18,19].Among young people, research revealed also a meaningful relationship between GHL and health behavior [20] as well as various health outcomes [21,22].Nonetheless, empowering adolescents to deal with health information requires sufficient GHL proficiency [23], and the same inevitably applies to DHL in a digitized era of health information and the health system [24,25].DHL is also referred to as the individual, social, and technical competencies that are essential for digitally searching, finding, understanding, and using health information [6,26].This has a number of potential consequences for individuals' health in the broadest sense, as behavior in the digital spaces requires particular skills, in terms of digital health literacy, to be able to promote and maintain health and wellbeing, in addition to prevent or deal with illnesses.
Recently, a scoping review to identify available tools to measure DHL [15], which was built on the concurrently existing research [27], revealed that eHEALS [28] was the most commonly used measure of DHL.Notably, the eHEALS measure was originally developed and validated among adolescents aged 13-21.However, the eHEALS was mostly identified as a two-factor scale when applied to young people [29].Summing up scores of individual items into a total score of DHL requires a unidimensional scale [30].Further, Faux-Nightingale et al. [31] suggested that eHEALS does not provide any fixed assessment or score to indicate, for instance, people's ability to interact with digital health-related resources.
Accordingly, the HLS 19 -Consortium of M-POHL developed a parsimonious unidimensional HLS 19 -DIGI scale for measuring people's DHL [6], hereafter called HLS 19 -DIGI.The development was conceptually based on the comprehensive concept and definition of GHL but is expanded to include the characteristics of digital information sources [6].In this study, we provisionally refer to this instrument as a measure for DHL among adult populations as it had not been validated for use among adolescents.
To our knowledge, no other unidimensional DHL instruments have been properly validated in adolescents aged 16 years and over [31].In addition, in the Health Literacy Tool Shed database we did not succeed in finding any instruments specifically for DHL, that were validated for use among adolescents [32].Meanwhile, eHEALS is another commonly used eHealth literacy instrument [28].As it was mostly identified as a two-factor scale with no fixed score to indicate a proficiency [31], its applicability has weakened [29,31].Consequently, our study aimed to investigate digital health literacy in young people using the newly developed instrument HLS 19 -DIGI and explore its association with health system navigating abilities.This was operationalized by (1) evaluating the psychometric properties of the HLS 19 -DIGI applied in adolescents and young adults aged 16-25 years; (2) describing the distribution of DHL proficiency for the target group; and finally, (3) exploring the associations between DHL and adolescents' NAV-HL.

Sampling and data collection
This study used data from the Norwegian part of Health Literacy Survey 2019-2021 (HLS 19 ) [5], that was collected using computer assisted telephone interviews during April-October 2020.The data collection was carried out in two waves and stratified by age groups (x8), genders (x2), and counties (x11), of which the data used in this study included 2 out of 8 age groups (16-17 and 18-25).A detailed description of the data collection can be found in Le et al. [33].Out of 6000 participants, 890 participants met our inclusion criteria "adolescents aged 16-25".Notwithstanding, the NAV-HL data (n = 471) was collected only in the second wave of the data collection.In Norway, there are health clinics for adolescents aged 13-20, but many clinics treat people even up to 25 years old.In this study, we therefore labeled the sampled population as "adolescents" and may interchangeably refer to the terms "young people" and "young adults".

Measures
The HLS 19 -DIGI instrument that measures DHL, consists of eight items measuring the ability to search for, access, understand, appraise, and apply online health information, for instance, "how easy or difficult is it to use the proper words or search query to find the information you are looking for?", or "…to judge whether the information is applicable to you?".The instrument uses a 4-point rating scale with the response categories; (1) very difficult, (2) difficult, (3) easy, and (4) very easy.A "don't know" response category was used when stated spontaneously by the participants, which was recoded to missing data in the analyses.
In combination with the HLS 19 -DIGI instrument, we also collected data on NAV-HL (full text/description in Table 1) based on the Norwegian version [5] of the HLS 19 -NAV scale of M-POHL [7], and sociodemographic factors such as age, gender, education, self-perceived social status, and self-reported financial deprivation.All variables were dichotomized in the analyses except for DHL proficiency.
Self-perceived social status was measured using: ["On the following scale, step '1' corresponds to "the lowest level in the society"; step '10' corresponds to "the highest level in the society".Could you tell me, how would you rank yourself?"],and self-reported financial deprivation was measured using: ["On a scale from "very easy" to "very difficult", how easy is it for you to pay all bills at the end of the month?"].The latter measure may be understood as "relative" as the youngest age group may be more dependent on the socioeconomic status of their parents, which would indirectly affect the 16 year-old adolescent's ability to "pay the bills at the end of the month".Therefore, we believe this factor was as relevant to the 16 yearold as it was to the older participants.
Other statistical analyses Independent samples t-test was used to compare the mean score of HLS 19 -DIGI between two independent groups, while the chi-square test was used to explore the differences of DHL (levels) across several sociodemographic factors (binary/categorical variables).Binary logistic regression models with dichotomized NAV-HL items as the dependent variables and DHL proficiency (raw score being transformed into person-location estimates in terms of logit values) as the independent variable were conducted.We also applied the "Wright's method" as set forth in Guttersrud et al. [51] to estimate the levels of DHL proficiency.Statistical significance was set at 5% level.

Characteristics of the participants
The study included 890 participants with a slight predominance of males (Table 1).Due to different waves of data collection [33], a smaller sample (n = 471) was applied to the variables concerning NAV-HL.Almost 80% of the participants have an education level at or below upper secondary school.Two-thirds report belonging to upper social level and above three out of four report no economic deprivation.

Overall data-model fit and unidimensionality of the HLS 19 -DIGI scale
The HLS 19 -DIGI scale could be considered sufficiently unidimensional as the proportion of individuals with significantly different person-location estimates on the compared subscales is slightly above 5% (Table 2).
Although the reliability indexes (PSR, PSI, and Omega) displayed good internal consistency, the scale could be better targeted to the population applied, as the distribution of person locations were right -skewed compared to the item-threshold locations.This indicated that the items are perceived as quite easy relative to the target population (Fig. 1), showing a ceiling effect.The overall chi-square statistic was significant indicating problems at the individual item level (Table 2).In turn, this could be seen in relation to the t-value above 1.96 (item5: t-value = 2.3) in 2 How easy or difficult is it for you to pay all bills at the end of month?

Table 1 (continued)
and below displayed non-significant overall chi-square statistics.

Rasch analyses at item level for the HLS 19 -DIGI
Applying unidimensional Rasch modeling all items had acceptable infit values (Table 3) except for item 5, which had a T-value of 2.3 meaning that the item under-discriminated relative to the PCM.No disordered response categories were observed, neither items displaying differential item functioning (Table 3).Moreover, no residual correlations above 0.3 were observed, which indicates none-significant response dependence between any two items (Table 4).

Levels of digital health literacy proficiency
Applying the Wright method, three distinctive levels (1-3) of DHL were found starting from a cut-point of 19, 24, and 29 out of 32 (Table 5), with level 1 labeled as the lowest and level 3 as the highest, respectively.While 46% are at or above level 2, more than half of the respondents (54%) are at or below level 1.People who achieved level 1 would typically perceive it as "easy" to use the proper words or search query to find the information they are looking for (item1) and to visit different websites to check whether they provide similar information about the topic (item6).Based on the content of these items, we can generalize that people who scored 19 or above probably can use digital resources to access quality assured information.People who achieved level 2 (sum score of 24 or above) would typically perceive the rest of the items as "easy", which are item2 [to find the exact information you are searching for], item3 [to understand the information], item4 [to judge whether the information is reliable], item5 [to judge whether the information is offered with commercial interests], item7 [to judge whether the information is applicable to you], and item8 [to use the information to help solve a health problem].This may be generalized that people who gained a score of 24 or above probably are able to understand, appraise, and apply the digital information accessed.

Abilities to navigate the healthcare system by levels of digital health literacy
More than 38% (40% excluding missing) of the participants report (very) difficult on NAV3, NAV4, NAV5, NAV6, and NAV8 (Table 1).Adjusted for age, gender, education, self-perceived social status, and self-reported financial deprivation, the results in Table 6 show that DHL is associated with all these NAV-HL items (b ranged from 0.07 to 0.11, p < .001).We also observed that more than 50% (ranged 57-74%) of the target population who experienced health system navigating abilities as (very) difficult are located at or below level 1 of DHL proficiency (Fig. 2).

Summary
The findings suggested that the HLS 19 -DIGI has acceptable psychometric properties in adolescents.A high proportion of low DHL proficiency among adolescents may represent a significant public health concern as well as a resource and capacity challenge for the healthcare services.

Targeting and content validity
The HLS 19 -DIGI was developed based on a HL-related conceptual framework combined with a theory-based model by which the selection of items is justified.Consequently, the scale's content validity is considered attested.However, the HLS 19 -DIGI scale obtained a relatively high positive mean person location value meaning that the person proficiencies were located at a higher level than the average difficulty of the scale.This indicates that the items could be considered to be too easy for the     participants' proficiency.Therefore, the scale could benefit from adding items that may require more challenging skills.
To access health information adolescents are expected to actively use social media and digital platforms [23,52].Paradoxically, item 5 that represents the ability to judge whether the information is offered with commercial interests, tends to under-discriminate even though young people are often exposed to health information [2].Another study suggested that adolescents would prefer their family as information resources rather than social media platforms [3].This habit could have contributed to weakening adolescents' ability to distinguish between reliable health information and information that are provided due to commercial interests, which in turn may have caused item under-discrimination by weakening its ability to distinguish between respondents with low versus high proficiency.Previous research has demonstrated a correlation between GHL and DHL [8], and this may provide a basis for planning interventions for adolescents to develop health literacy skills enabling them to critically assess health information accessed [2].

Overall and individual item fit
The overall chi-square statistic was statistically significant, and that could be caused by item5 with the t-value above 1.96 (t-value = 2.3).Furthermore, chi-square statistics are very sensitive to a large sample size, implying that any small differences would mostly appear statistically significant [53].Meanwhile, a large sample size in Rasch modeling is not strictly required.A rule of thumb suggests sample size for a test of eight polytomous items with three thresholds should comprise at least 240 up to 480 persons, in which a reasonable ratio is between 10 and 20 persons for each threshold [47].Subsequently, when the sample size was at random gradually reduced to n = 305 the data displayed acceptable overall data-model fit in terms of non-significant overall chi-square statistics.

Digital health literacy and health system navigating abilities among adolescents
Only people at level 2 and above would typically be able to understand, appraise, and apply the digital health information they access.And only people at level 1 or above would be able to use digital resources to access quality assured information.Based on the "skills" that only people at level 2 and above would typically master, we can assume that people at level 1 and below have low and inadequate DHL proficiency.In particular, people below level 1 (12%) may have problematic DHL proficiency as they probably would not be able to use digital resources to access and quality assure health information that they may need.Moreover, analyses revealed that there is a statistically significant association between DHL and adolescents' NAV-HL.Consequently, low DHL indicates a considerable problem for properly navigating the health system and appropriately utilizing the health services available.This finding partially responds to previous research revealing that young people have less access to health care than adults [1].There are in total 54% of the target population located at or below level 1 of DHL proficiency.This may indicate a public health concern as well as a resource and capacity challenge for the health services.The results suggest more targeted efforts, both individually and structurally, to enhance adolescents' opportunity for better health and wellbeing.

Strength and limitations
Novelty is the strength of the study; to our knowledge, this is the first study that we know of that has validated a unidimensional scale (HLS 19 -DIGI) measuring DHL in adolescents.A population-based study -as far as it was feasible -using nationally representative strata is another rare strength covering the population studied, holistically.
The HLS 19 -DIGI scale was psychometrically assessed in adolescents based on national data.Hence, the psychometric properties of the instrument should be further assessed using other samples or data from other countries to generalize the conclusions of the study.In addition to estimating abilities for using search engines, mastering search strategies, and critically appraising sources and selection of relevant digital health information, DHL also involves the ability to utilize digital health services, which therefore includes people's general digital skills such as the ability to communicate via digital channels and solve technical problems entailed using digital devices.Consequently, further research on DHL should pay more attention to the latter two aspects; (1) users' readiness to utilize digital healthcare services and (2) users' general digital skills.
Note w weighted fit MNSQ, one-dimensional model using ConQuest 5; u A t-value > 1.96 indicates a poorly fitting item in terms of under-discrimination relative to the Rasch model DIF: differential item functioning; *Bonferroni-adjusted 5% has been used to assist detecting possible significant deviations due to DIF; Ordered: "yes" refers to an item with none disordered response categories F = Find; U = Understand; J = Judge; A = Apply; lb: lower bound; ub: upper bound; CI = confidence interval; T-value = similar to the z standardized fit statistics in unidimensional Rasch analyses; MNSQ = mean square value; SE = Standard Error

Fig. 2
Fig. 2 Abilities to navigate the healthcare system in light of levels of digital health literacy.Note: Percentage of difficult/very difficult without missing

Table 1
Digital health literacy (in logits) using the HLS 19 -DIGI by sociodemographic characteristics and health system navigating abilities (NAV-HL) Table 3 that indicates a poorly fitting item in term of under-discrimination relative to the Rasch model.However, a smaller sample size that at random was gradually reduced to n = 305 Note DHL: Standardized score (Person location estimates in logits) of digital health literacy (DHL) by means of the HLS 19 -DIGI.Higher values indicate higher DHL 1 categorical data: frequencies, n (percentage [%]); continuous data: mean (standard deviation [sd])

Table 2
Unidimensionality, overall data-model fit, and reliability by applying Rasch modeling of the HLS 19 -DIGI Note * total item chi-square is significant at 5%-level indicating significant deviation between the observed data and what was expected from the Rasch model; dim(%): proportion of individuals with significantly different personlocation estimates (below 5% confirms unidimensionality); proportion lower 95% CI: lower than 5% confirms acceptable unidimensionality; df: degree of freedom; SD: Standard deviation; SE: Standard error; Omega: internal consistency reliability; PSI: person separation index; PSR: person separation reliability; PMLE: pairwise maximum likelihood estimate; MMLE: marginal maximum likelihood estimate; WLE: Warm's mean likelihood estimate; Deviance: deviance statistics; ep: total number of estimated parameters; AIC: Akaike Information Criterion; RUMM: RUMM2030 software; CQ/ConQuest: ConQuest 5 software

Table 3
Item characteristics, disordered response categories, and DIF of the HLS 19 -DHL Item: On a

Table 4
Residual correlations and response dependence between any two items of the HLS 19 -DIGI

Table 5
Proportion of participants at three statistically distinct digital health literacy levels (HLS 19 -DIGI cut-points) across person factors

Table 6
Binary logistic regression with NAV-abilities by DHL and covariates

OR) p-value Coef. (OR) p-value Coef. (OR) p-value Coef. (OR) p-value Coef. (OR) p-value
Note Only five NAV-variables that have over 40% of the "difficult"-category (excluding missing count) located at level 1 and below, have been included in the report of logistic regression analyses Coef.= unstandardized regression slope/coefficient; OR: odds ratio NAV3: ability to judge to what extent a health insurance covers your need of a particular health service NAV4: ability to find out if a particular healthcare service requires a deductible